CORRESPONDENCE 


Infect Dis Clin Microbiol 2024; 6(2): 151-3 


ChatGPT as a Novel Consultant in Infectious 


Diseases and Clinical Microbiology 


Hinpetch Daungsupawong"(), Viroj Wiwanitkit? @® 


Private Academic Consultant, Phonhong, Lao People's Democratic Republic 


2Saveetha Medical College, Saveetha Institute of Medical and Technical Sciences, Chennai, India 


To the Editor, 

Tuncer et al. brought us interesting observations from the study “How Reliable is 
ChatGPT as a Novel Consultant in Infectious Diseases and Clinical Microbiology?” (1). 
In conclusion, 200 questions about infectious diseases from different platforms were 
used in the study, along with recommendations from reliable sources. The replies were 
predicated on predetermined standards, and the questions were meticulously selected 
and arranged for coherence and clarity. The study's scoring mechanism made it possible 
to evaluate ChatGPT's precision in giving accurate responses to every query. Overall, 
ChatGPT performed well, according to the results, with inquiries about urinary tract 
infections showing the lowest accuracy. 


However, the study’s findings also revealed potential areas for improvement. For in- 
stance, there was a notable discrepancy in accuracy rates between questions sourced 
from guidelines and those from social media sites. When it came to appropriately an- 
swering queries on social media platforms, ChatGPT performed better than guidelines 
in terms of response rate. This disparity could indicate a limitation in the model's ca- 
pacity to comprehend and deliver precise data in line with accepted standards. Sub- 
sequent investigations could concentrate on resolving this disparity and enhancing 
ChatGPT's efficacy with regard to questions that follow guidelines. 


Furthermore, method faults in the study may include biases in the selection of ques- 
tions from social media platforms and standards. The study may have needed a broad 
and representative sample of questions from both sources, which could have biased the 
findings. Future research could benefit from a more methodical approach to question 
selection to ensure a balanced representation of topics and sources. 


Looking ahead, future research could focus on fine-tuning the study's grading system 
to account for variations in model responses. This could contribute to a more nuanced 
assessment of ChatGPT's performance and potentially enhance its ability to provide 
accurate responses across a wider range of inquiries. Moreover, exploring strategies 
to improve ChatGPT's accuracy in responding to guideline-based questions could be a 
valuable area of investigation, with the aim of optimizing its utility in providing accurate 
and reliable information on infectious diseases. 
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The Authors Reply 

We appreciate Daungsupawong and Wiwanitkitet’s 
interest in our study. They only point out the sig- 
nificant difference in ChatGPT’s correct response 
rates between guidelines and social media ques- 
tions. They argue that this could be caused by ques- 
tion selections. We agree that bias is possible when 
choosing the questions, especially from social me- 
dia. However, we want to emphasize that we metic- 
ulously selected the questions derived from social 
media, taking care to reduce the risk of selection 
bias. We, as specialists, evaluated these questions 
for grammatical and linguistic clarity. Moreover, 
when we evaluate the methodology of the studies 
on the performance of artificial intelligence tools, 


Gllsah Tuncer®, Kadir Gorkem Guclu2@ 


it is seen that similar methods have been used for 
selecting questions (1-3). 


Because ChatGPT mostly uses publicly available of- 
ficial websites such as the World Health Organiza- 
tion (WHO) or the Centers for Disease Control and 
Prevention (CDC), which focus on public health in- 
formatics, it has accurately answered the questions 
derived from social media. As a result, since the 
guidelines are directed to experts, the questions de- 
rived from the guidelines are more complex. Asking 
professional questions results in poor performance. 
Therefore, we found a natural and expected result 
that ChatGPT is more capable of answering social 
media questions than guideline questions. 
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